Recent years have seen a surge in the number and diversity of wireless and mobile network applications. A key differentiator among such applications may rest in their ability to deliver a consistent and reliable service to the user. Such delivery through mobile applications is a significant challenge, however, as it requires the coordination of a range of key components in the end-to-end mobile ecosystem. More particularly, the components of the ecosystem, including, for example, the mobile device, a mobile network, and an application server, may exhibit significant variations in their behavior.
In order to ensure reliability, a mobile application may therefore need to be tested against each or all of the variations exhibited by these components. In general, device variation may be addressed through device diversity testing, while application servers are subjected to load and performance testing. However, the network component is harder to address, as it may include many factors outside the control of the application, such as network connectivity, bandwidth, and network congestion.
A key requirement of application performance is to be resilient to such external and environmental disturbances, which, consequently, imposes a need for testing in all possible network conditions. Such testing can be done by means of network simulation, network emulation, live network testing, or a combination thereof. While simulation offers significant benefits in controllability of the network environment, network simulation is ultimately a best-guess or approximation of live network conditions, and may fail to account for the full extent of non-deterministic network behavior. In addition, live testing, which is testing carried out in an actual live network environment, as its name indicates, does present test-cases that closely approximate real-world use cases, but the reproducibility of test-cases is problematic, given the lack of controllability over the network environment.
In contrast, the emulation approach charts a middle path between these two scenarios. In the context of testing wireless applications, the real application runs on the actual device. The network stack, on the actual emulator device, is also real. Given this setup, network behavior may be controlled by the emulator, which decides what packets the wireless mobile device sends and receives.
Mere repetition of a test case supported by a network emulator, however, may not serve to ensure test result repeatability due to the fact that test cases of a statistical nature may create a variety of different and unpredictable network events, such as a random packet drop, when executed at different points of time, thus leading to different test results.
Accordingly, there is a need for a network emulator mechanism whereby an ability to emulate network conditions and reproduce recorded test results at any point of time is preserved while accounting for a variation in incoming traffic and random network events.